A. Field of the Invention
The present invention is directed to the field of video coding and, more specifically, to scalable video coding.
B. Background
In some scenarios, it is desirable to transmit an encoded digital video sequence at some minimum or “base” quality, and in concert transmit an “enhancement” signal that may be combined with the minimum quality signal in order to yield a higher-quality decoded video sequence. Such an arrangement simultaneously allows some decoding of the video sequence by devices supporting some set of minimum capabilities (at the “base” quality), while enabling other devices with expanded capability to decode higher-quality versions of the same sequence, without incurring the increased cost associated with transmitting two independently coded versions of the same sequence.
In some situations, more than two levels of quality may be desired. In this case, multiple “enhancement” signals can be transmitted, each building on the “base” quality signal plus all lower-quality “enhancement” signals. Such “base” and “enhancement” signals are referred to as “layers” in the filed of scalable video coding, and the degree to which each enhancement layer improves on the reconstructed quality of the signal is referred to as the “granularity.” Fine granularity scalability (FGS) is a type of scalability in which the incremental quality increases provided by each layer are relatively small.
Various FGS techniques are known and used. For example, in the context of MPEG-21 SVC, a block-based FGS scheme has been proposed. In this scheme, decoding of each FGS layer is divided into three passes: the “significance pass,” the “refinement pass,” and the “remainder pass.” The significance pass identifies those coefficients that had previous reconstructed values of zero in the previous bit plane, and which had one or more neighboring coefficients with a non-zero reconstructed value in the previous bit plane. An encoded binary digit serves as a “significance bit” indicating whether the coefficient transitions from zero to non-zero in the current bit plane. The refinement pass identifies those coefficients that had reconstructed non-zero values in the previous bit plane. An encoded binary digit refines the precision of these coefficients in the current bit plane. The remainder pass encodes the remaining coefficients (i.e. those not already identified in the first or second passes). A “significance bit” is encoded for each coefficient, just as in the significance pass, however the transition from zero to non-zero is statistically less likely in the absence of neighboring non-zero values, thus justifying a separate pass for this category of non-zero coefficients.
If the video sequence involves decoding multiple components in a pass (such as both luminance and chrominance, written as YUV or YCbCr), the scheme proposed requires that, within each pass, all components must be completely decoded before moving on to the next pass. For example, the significance pass must be completed for both luminance and chrominance before the refinement pass can begin for either component.
In this proposed scheme, passes are arranged in subbands. For example, the significance pass can be described using the following pseudo-code:
For each subbandFor each blockDecode one luminance coefficientFor each blockDecode one coefficient from each chrominance componentGo to refinement pass
Note that the algorithm fixes the relative start positions of the luminance and chrominance components, i.e. the first chrominance subband always occurs between the first and second luminance subbands. In addition, the “speed” at which a pass is completed is constant across the color components. For example, after the significance pass is 50% complete, both luminance and chrominance components will be 50% decoded.
A second scheme has been proposed which seeks to solve some of the problems associated with such a rigid structure. In this scheme, the number of coefficients coded from each block is not fixed. Pseudo-code for the significance pass for this scheme can be written as:
While values remain to be decodedFor each blockDecode one non-zero luminance coefficient and preceding zerosDecode one non-zero chrominance coefficient from eachcomponent and preceding zerosGo to refinement pass
This scheme better interleaves luminance and chrominance values, since the “period” between non-zero values from a given component is a smaller unit (i.e. a block rather than a subband). However, again the entire significance pass must be completed for all components before moving to the remainder pass.
There are several problems with each of these approaches. For example, these schemes can suffer from coding inefficiency problems and do not provide much flexibility or control over the encoding/decoding process. As such, there is a need for an improved FGS coder that is more flexible than previous proposed schemes. There is also a need for a FGS coding scheme that provides an overall improvement in coding efficiency.